DevOps Enterprise Summit Virtual – US

The National Weather Service (NWS) provides weather forecasts for US and global communities using a suite of complex numerical models run on HPC platforms operated by the US Government's National Oceanic and Atmospheric Agency, NOAA. Weather Service products range from high resolution, hourly forecasts for the aviation community to months long forecasts for resilience management. Our models have historically been developed internally and run on internal platforms. Over the last five years we have embarked on an ambitious project to transform the development environment for these models from an in-house, stovepiped approach to one that embraces open community modeling. Our models are already starting to benefit from contributors beyond our traditional partners even as we are starting on this journey; one initial result is that by implementing parallel compression algorithms our model I/O footprint has reduced six fold, allowing us to increase resolution and improve forecast skills.

In this talk I will briefly introduce our modeling suites and walk through our efforts to change NOAA’s model development culture and modernize our suites. I will talk about the challenges we faced in embracing DEV-OPS processes that include testing and development on a range of HPC platforms both within and outside NOAA. Finally, as cloud adoption is speeding up, I will close with our efforts to successfully migrate models that require massively parallel distributed memory tasks with fast networks to cloud infrastructure.